Deep Hybrid Learning Prediction of Patient-Specific Quality Assurance in Radiotherapy: Implementation in Clinical Routine.

BACKGROUND: Arc therapy allows for better dose deposition conformation, but the radiotherapy plans (RT plans) are more complex, requiring patient-specific pre-treatment quality assurance (QA). In turn, pre-treatment QA adds to the workload. The objective of this study was to develop a predictive model of Delta4-QA results based on RT-plan complexity indices to reduce QA workload. METHODS: Six complexity indices were extracted from 1632 RT VMAT plans. A machine learning (ML) model was developed for classification purpose (two classes: compliance with the QA plan or not). For more complex locations (breast, pelvis and head and neck), innovative deep hybrid learning (DHL) was trained to achieve better performance. RESULTS: For not complex RT plans (with brain and thorax tumor locations), the ML model achieved 100% specificity and 98.9% sensitivity. However, for more complex RT plans, specificity falls to 87%. For these complex RT plans, an innovative QA classification method using DHL was developed and achieved a sensitivity of 100% and a specificity of 97.72%. CONCLUSIONS: The ML and DHL models predicted QA results with a high degree of accuracy. Our predictive QA online platform is offering substantial time savings in terms of accelerator occupancy and working time.

Influence of inputs for bone lesion segmentation in longitudinal 18F-FDG PET/CT imaging studies.

In metastatic breast cancer, bone metastases are prevalent and associated with multiple complications. Assessing their response to treatment is therefore crucial. Most deep learning methods segment or detect lesions on a single acquisition while only a few focus on longitudinal studies. In this work, 45 patients with baseline (BL) and follow-up (FU) images recruited in the context of the EPICUREseinmeta study were analyzed. The aim was to determine if a network trained for a particular timepoint can generalize well to another one, and to explore different improvement strategies. Four networks based on the same 3D U-Net framework to segment bone lesions on BL and FU images were trained with different strategies and compared. These four networks were trained 1) only with BL images 2) only with FU images 3) with both BL and FU images 4) only with FU images but with BL images and bone lesion segmentations registered as input channels. With the obtained segmentations, we computed the PET Bone Index (PBI) which assesses the bone metastases burden of patients and we analyzed its potential for treatment response evaluation. Dice scores of 0.53, 0.55, 0.59 and 0.62 were respectively obtained on FU acquisitions. The under-performance of the first and third networks may be explained by the lower SUV uptake due to treatment response in FU images compared to BL images. The fourth network gives better results than the second network showing that the addition of BL PET images and bone lesion segmentations as prior knowledge has its importance. With an AUC of 0.86, the difference of PBI between two acquisitions could be used to assess treatment response. Clinical relevance- To assess the response to treatment of bone metastases, it is crucial to detect and segment them on several acquisitions from a same patient. We proposed a completely automatic method to detect and segment these metastases on longitudinal 18F-FDG PET/CT images in the context of metastatic breast cancer. We also proposed an automatic PBI to quantitatively assess the evolution of the bone metastases burden of patient and to automatically evaluate their response to treatment.

Deformable image registration with deep network priors: a study on longitudinal PET images.

Objective.This paper proposes a novel approach for the longitudinal registration of PET imaging acquired for the monitoring of patients with metastatic breast cancer. Unlike with other image analysis tasks, the use of deep learning (DL) has not significantly improved the performance of image registration. With this work, we propose a new registration approach to bridge the performance gap between conventional and DL-based methods: medical image registration method regularized by architecture (MIRRBA).Approach.MIRRBAis a subject-specific deformable registration method which relies on a deep pyramidal architecture to parametrize the deformation field. Diverging from the usual deep-learning paradigms,MIRRBAdoes not require a learning database, but only a pair of images to be registered that is used to optimize the network's parameters. We appliedMIRRBAon a private dataset of 110 whole-body PET images of patients with metastatic breast cancer. We used different architecture configurations to produce the deformation field and studied the results obtained. We also compared our method to several standard registration approaches: two conventional iterative registration methods (ANTs and Elastix) and two supervised DL-based models (LapIRN and Voxelmorph). Registration accuracy was evaluated using the Dice score, the target registration error, the average Hausdorff distance and the detection rate, while the realism of the registration obtained was evaluated using Jacobian's determinant. The ability of the different methods to shrink disappearing lesions was also computed with the disappearing rate.Main results.MIRRBA significantly improved all metrics when compared to DL-based approaches. The organ and lesion Dice scores of Voxelmorph improved by 6% and 52% respectively, while the ones of LapIRN increased by 5% and 65%. Regarding conventional approaches, MIRRBA presented comparable results showing the feasibility of our method.Significance.In this paper, we also demonstrate the regularizing power of deep architectures and present new elements to understand the role of the architecture in DL methods used for registration.

Speaking with virtual humans: Assessing social cognition in traumatic brain injury with a second-person perspective task.

OBJECTIVE: This study is one of the first to investigate social cognition in participants with traumatic brain injury (TBI) using a task that actively engaged the participant in a real interaction with a partner. Previous results have reported altered social cognition in TBI patients, but social cognition was mostly assessed through traditional tasks involving conscious and deliberate reasoning about characters' mental states (i.e., a third-person perspective). Our goal was to present a new paradigm which allowed the assessment of social cognition in conditions closer to real life meaning that participants were actively engaged in an interaction (i.e., second-person perspective) in order to capture more implicit use of social cognition processes. METHOD: This study used three tasks to evaluate social cognition. We designed a task, called EVICog, in which participants were engaged in real audio-visual conversations with two virtual humans who expressed emotions and produced speech content that required the participants to make inferences about the characters' mental states. The two other tasks are standard in the literature; they use photographs to test participants' recognition of emotions and short comic strips to test their attribution of intentions. RESULTS: Our results showed that TBI participants presented a significant deficit of social cognition compared to control participants. The ROC analysis showed that EVICog has a high discrimination power compared to the other tests. CONCLUSION: These results further confirm that social cognition is altered in TBI participants even in real interactions and further support the use of ecological settings to investigate social cognition.

Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment.

Metastatic breast cancer patients receive lifelong medication and are regularly monitored for disease progression. The aim of this work was to (1) propose networks to segment breast cancer metastatic lesions on longitudinal whole-body PET/CT and (2) extract imaging biomarkers from the segmentations and evaluate their potential to determine treatment response. Baseline and follow-up PET/CT images of 60 patients from the EPICUREseinmeta study were used to train two deep-learning models to segment breast cancer metastatic lesions: One for baseline images and one for follow-up images. From the automatic segmentations, four imaging biomarkers were computed and evaluated: SULpeak, Total Lesion Glycolysis (TLG), PET Bone Index (PBI) and PET Liver Index (PLI). The first network obtained a mean Dice score of 0.66 on baseline acquisitions. The second network obtained a mean Dice score of 0.58 on follow-up acquisitions. SULpeak, with a 32% decrease between baseline and follow-up, was the biomarker best able to assess patients' response (sensitivity 87%, specificity 87%), followed by TLG (43% decrease, sensitivity 73%, specificity 81%) and PBI (8% decrease, sensitivity 69%, specificity 69%). Our networks constitute promising tools for the automatic segmentation of lesions in patients with metastatic breast cancer allowing treatment response assessment with several biomarkers.

Deep learning approaches for bone and bone lesion segmentation on 18FDG PET/CT imaging in the context of metastatic breast cancer.

18FDG PET/CT imaging is commonly used in diagnosis and follow-up of metastatic breast cancer, but its quantitative analysis is complicated by the number and location heterogeneity of metastatic lesions. Considering that bones are the most common location among metastatic sites, this work aims to compare different approaches to segment the bones and bone metastatic lesions in breast cancer.Two deep learning methods based on U-Net were developed and trained to segment either both bones and bone lesions or bone lesions alone on PET/CT images. These methods were cross-validated on 24 patients from the prospective EPICUREseinmeta metastatic breast cancer study and were evaluated using recall and precision to measure lesion detection, as well as the Dice score to assess bones and bone lesions segmentation accuracy.Results show that taking into account bone information in the training process allows to improve the precision of the lesions detection as well as the Dice score of the segmented lesions. Moreover, using the obtained bone and bone lesion masks, we were able to compute a PET bone index (PBI) inspired by the recognized Bone Scan Index (BSI). This automatically computed PBI globally agrees with the one calculated from ground truth delineations.Clinical relevance- We propose a completely automatic deep learning based method to detect and segment bones and bone lesions on 18FDG PET/CT in the context of metastatic breast cancer. We also introduce an automatic PET bone index which could be incorporated in the monitoring and decision process.

Combining Superpixels and Deep Learning Approaches to Segment Active Organs in Metastatic Breast Cancer PET Images.

Semi-automatic measurements are performed on 18FDG PET-CT images to monitor the evolution of metastatic sites in the clinical follow-up of metastatic breast cancer patients. Apart from being time-consuming and prone to subjective approximation, semi-automatic tools cannot make the difference between cancerous regions and active organs, presenting a high 18FDG uptake.In this work, we combine a deep learning-based approach with a superpixel segmentation method to segment the main active organs (brain, heart, bladder) from full-body PET images. In particular, we integrate a superpixel SLIC algorithm at different levels of a convolutional network. Results are compared with a deep learning segmentation network alone. The methods are cross-validated on full-body PET images of 36 patients and tested on the acquisitions of 24 patients from a different study center, in the context of the ongoing EPICUREseinmeta study. The similarity between the manually defined organ masks and the results is evaluated with the Dice score. Moreover, the amount of false positives is evaluated through the positive predictive value (PPV).According to the computed Dice scores, all approaches allow to accurately segment the target organs. However, the networks integrating superpixels are better suited to transfer knowledge across datasets acquired on multiple sites (domain adaptation) and are less likely to segment structures outside of the target organs, according to the PPV.Hence, combining deep learning with superpixels allows to segment organs presenting a high 18FDG uptake on PET images without selecting cancerous lesion, and thus improves the precision of the semi-automatic tools monitoring the evolution of breast cancer metastasis.Clinical relevance- We demonstrate the utility of combining deep learning and superpixel segmentation methods to accurately find the contours of active organs from metastatic breast cancer images, to different dataset distributions.

Effects of cognitive impairment on prosodic parameters of speech production planning in multiple sclerosis.

Cognitive impairment (CI) affects 40-65% of patients with multiple sclerosis (MS). CI can have a negative impact on a patient's everyday activities, such as engaging in conversations. Speech production planning ability is crucial for successful verbal interactions and thus for preserving social and occupational skills. This study investigates the effect of cognitive-linguistic demand and CI on speech production planning in MS, as reflected in speech prosody. A secondary aim is to explore the clinical potential of prosodic features for the prediction of an individual's cognitive status in MS. A total of 45 subjects, that is 22 healthy controls (HC) and 23 patients in early stages of relapsing-remitting MS, underwent neuropsychological tests probing specific cognitive processes involved in speech production planning. All subjects also performed a read speech task, in which they had to read isolated sentences manipulated as for phonological length. Results show that the speech of MS patients with CI is mainly affected at the temporal level (articulation and speech rate, pause duration). Regression analyses further indicate that rate measures are correlated with working memory scores. In addition, linear discriminant analysis shows the ROC AUC of identifying MS patients with CI is 0.70 (95% confidence interval: 0.68-0.73). Our findings indicate that prosodic planning is deficient in patients with MS-CI and that the scope of planning depends on patients' cognitive abilities. We discuss how speech-based approaches could be used as an ecological method for the assessment and monitoring of CI in MS.

Determinants of theory of mind performance in Alzheimer's disease: A data-mining study.

Whether theory of mind (ToM) is preserved in Alzheimer's disease (AD) remains a controversial subject. Recent studies have showed that performance on some ToM tests might be altered in AD, though to a lesser extent than in behavioural-variant Frontotemporal Dementia (bvFTD). It is however, unclear if this reflects a genuine impairment of ToM or a deficit secondary to the general cognitive decline observed in AD. Aiming to investigate the cognitive determinants of ToM performance in AD, a data-mining study was conducted in 29 AD patients then replicated in an independent age-matched group of 19 AD patients to perform an independent replication of the results. 44 bvFTD patients were included as a comparison group. All patients had an extensive neuropsychological examination. Hierarchical clustering analyses showed that ToM performance clustered with measures of executive functioning (EF) in AD. ToM performance was also specifically correlated with the executive component extracted from a principal component analysis. In a final step, automated linear modelling conducted to determine the predictors of ToM performance showed that 48.8% of ToM performance was significantly predicted by executive measures. Similar findings across analyses were observed in the independent group of AD patients, thereby replicating our results. Conversely, ToM impairments in bvFTD appeared independent of other cognitive impairments. These results suggest that difficulties of AD patients on ToM tests do not reflect a genuine ToM deficit, rather mediated by general (and particularly executive) cognitive decline. They also suggest that EF has a key role in mental state attribution, which support interacting models of ToM functioning. Finally, our study highlights the relevancy of data-mining statistical approaches in clinical and cognitive neurosciences.

Theory of mind in Alzheimer disease: Evidence of authentic impairment during social interaction.

OBJECTIVE: The present study aimed to investigate theory of mind (the ability to infer others' mental states) deficit in 20 patients with mild Alzheimer's disease and 20 healthy controls, with 2 theory of mind tasks, 1 of them being a real interactive task. Previous results concerning preserved or altered theory of mind abilities in Alzheimer's disease have been inconsistent and relationships with other cognitive dysfunctions (notably episodic memory and executive functions) are still unclear. METHOD: The first task we used was a false belief paradigm as frequently used in literature whereas the second task, a referential communication task, assessed theory of mind in a real situation of interaction. Participants also underwent neuropsychological evaluation to investigate potential relationships between theory of mind and memory deficits. RESULTS: The results showed that Alzheimer patients presented a genuine and significant theory of mind deficit compared to control participants characterized notably by difficulties to attribute knowledge to an interlocutor in a real social interaction. CONCLUSION: These results further confirm that theory of mind is altered in early stages of Alzheimer dementia which is consistent with previous works. More specifically, this study is the first to objectivize this impairment in social interaction.

Different Patterns of Theory of Mind Impairment in Mild Cognitive Impairment.

Theory of Mind refers to the ability to infer other's mental states, their beliefs, intentions, or knowledge. To date, only two studies have reported the presence of Theory of Mind impairment in mild cognitive impairment (MCI). In the present study,we evaluated 20 MCI patients and compared them with 25 healthy control participants using two Theory of Mind tasks. The first task was a false belief paradigm as frequently used in the literature, and the second one was a referential communication task,assessing Theory of Mind in a real situation of interaction and which had never been used before in this population. The results showed that MCI patients presented difficulties inferring another person's beliefs about reality and attributing knowledge to them in a situation of real-life interaction. Two different patterns of Theory of Mind emerged among the patients. In comparison with the control group, some MCI patients demonstrated impairment only in the interaction task and presented isolated episodicmemory impairment, while others were impaired in both Theory of Mind tasks and presented cognitive impairment impacting both episodic memory and executive functioning. Theory of Mind is thus altered in the very early stages of cognitive impairment even in real social interaction, which could impact precociously relationships in daily life.

Characteristics of current tasks that contribute to mentalizing judgments: does the engagement of the participants in the social interaction matter? Comment on Achim et al. (2013).

In a recent article, Achim et al. (2013) discussed the different sources of information that contribute to mentalizing judgments in current theory-of-mind (ToM) tasks. The authors rightly emphasized the dynamic aspect of real-life social interaction, suggesting that taking account of the ongoing changes occurring during social interaction would make ToM tasks more ecological. They proposed a framework (i.e., the Eight Sources of Information Framework) that specifies the 8 sources of information we get from the environment and/or from our memories to attribute mental states to others. Nevertheless, we believe that a central aspect of ToM is missing in this framework: the engagement (or not) of the participant in the social interaction during ToM assessment. Indeed, this framework fails to consider how the participant who takes part in the ToM task manages this information, depending on the fact that he or she is involved in the interaction or not and how the information concerning the agent may impact the participant attribution of mental states. We reviewed several arguments and results from the ToM literature suggesting that merely observing a social interaction is not equivalent to participating in an interaction in terms of cognitive processes involved in the attribution of mental states to others.

Using memories to understand others: the role of episodic memory in theory of mind impairment in Alzheimer disease.

Theory of mind (TOM) refers to the ability to infer one's own and other's mental states. Growing evidence highlighted the presence of impairment on the most complex TOM tasks in Alzheimer disease (AD). However, how TOM deficit is related to other cognitive dysfunctions and more specifically to episodic memory impairment - the prominent feature of this disease - is still under debate. Recent neuroanatomical findings have shown that remembering past events and inferring others' states of mind share the same cerebral network suggesting the two abilities share a common process .This paper proposes to review emergent evidence of TOM impairment in AD patients and to discuss the evidence of a relationship between TOM and episodic memory. We will discuss about AD patients' deficit in TOM being possibly related to their difficulties in recollecting memories of past social interactions.